Decoding the visual attention of an individual from electroencephalographic signals

ABSTRACT

A method for determining the focus of the visual attention of an individual from electroencephalographic signals. At least one visual stimulus to be displayed is generated from at least one graphical object, a visual stimulus being an animated graphical object obtained by applying to a graphical object a temporal succession of elementary transformations that is temporally parameterized by a corresponding modulation signal. From a plurality of electroencephalographic signals produced by the individual focusing his visual attention to one of the visual stimuli, a modulation signal is reconstructed. A visual stimulus corresponding to the modulation signal for which the degree of statistical dependence with the reconstructed modulation signal is higher than a first threshold is identified.

TECHNICAL FIELD

The present description relates to a method and system for determiningthe focus of visual attention of an individual fromelectroencephalographic signals.

PRIOR ART

The emergence of various portable systems dedicated to recording andexploiting electroencephalographic signals (EEG signals) in multipleapplications is observable. In particular, the miniaturization ofsystems for recording EEG signals and substantial developments in theanalysis techniques used for real-time decoding of EEG signals meansthat new applications may now be envisioned that are both rapid andreliable to use.

Certain decoding techniques are based on an extraction ofelectro-physiological features from the EEG signals that allowpredicting the ongoing relationship between brain activity and visualstimuli in the environment. The difficulty here consists in identifying,in the EEG signal and in real-time, the visual stimulus specificfeatures belonging to the stimulus an individual is attending among amultitude of other visual inputs. Such decoding must be robust, i.e.must allow determining which specific content the individual isattending, in order to be able to trigger a command corresponding to thevisual stimulus, with sufficient speed and precision.

Patent document U.S. Pat. No. 8,391,966B2 describes a technique foranalyzing EEG signals produced by an individual observing stimuli, eachstimulus consisting of a light source flashing at a given frequency.Various features are generated in order to classify the EEG signals intoclasses corresponding to the different stimuli, with a view toidentifying the visual stimulus observed at a given time.

The EEG signals are for example divided into successive segments andcorrelation coefficients between pairs of segments of a given signal arecomputed in order to produce a first set of features. An averagecorrelation coefficient is computed then compared to a threshold inorder to determine whether the user is observing the stimulus or not.Moreover, the correlation between an EEG signal and the stimulus may beanalyzed in order to generate a second set of features: the degree ofcorrelation with a stimulus will be higher if the individual is actuallyobserving this stimulus. The coefficients of an autoregressive model maybe computed from an average EEG signal, the model's coefficient forminga third set of features.

This technique assumes prior classification of the EEG by means of aplurality of sets of features in association with a discriminatingmethod based on thresholding, a search for nearest neighbors, neuralnetworks, etc. The technique is therefore dependent on the relevance ofthe features used and the classification method employed.

Furthermore, the technique is limited to stimuli taking the form offlashing lights, greatly limiting the scope of application.

Other methods are known. For example, the document entitled “An onlinemulti-channel SSVEP-based brain computer interface using canonicalcorrelation analysis method” by Guangyu Bin et al, (Journal of NeuralEngineering, IOP Publishing, 2009, uses a modified canonical correlationanalysis (CCA) method.

Thus, there appears to be a need for a reliable and real-time EEGdecoding technique that is applicable to a brain-machine interfacebetween a user and a software application, for example one comprisingtext, images and/or menus.

SUMMARY

According to a first aspect, an object of the present description is amethod for determining the focus of an individual's visual attentionfrom electroencephalographic signals. The method comprises thegeneration of a set of at least one visual stimulus to be displayed fromat least one graphical object, from at least one elementarytransformation and from a set of at least one modulation signal, avisual stimulus being an animated graphical object obtained by applyingto a graphical object a temporal succession of elementarytransformations that is temporally parameterized by a correspondingmodulation signal; reconstructing a modulation signal from a pluralityof electroencephalographic signals produced by the individual in orderto obtain a reconstructed modulation signal; computing a degree ofstatistical dependence between the reconstructed modulation signal andeach modulation signal of the set of at least one modulation signal;identifying at least one visual stimulus corresponding to a modulationsignal for which the degree of statistical dependence is higher than afirst threshold.

The method according to the first aspect consists in a hybrid decodingmethod that combines an approach based on stimulus reconstruction fromelectroencephalographic (EEG) signals with a method of excitingutilizing one or more stimuli, each having temporal characteristics ableto be reproduced in the brain of an individual observing these one ormore stimuli, as these temporal characteristics can be found in the EEGsignals and therefore can be directly identified within the EEG signals.Such a combination makes it possible both to increase the sensitivity(i.e. the signal-to-noise ratio) of the EEG signal to the generatedstimuli and to provide a robust analysis method that is applicable inreal-time. The method is applicable to one or more visual stimuli.

Furthermore, the use of elementary transformations (for example avariation in light intensity, a variation in contrast, a colorimetrictransformation, a geometric deformation, a rotation, and oscillation, amovement along a path, etc.) allows an extensive range of visual stimulitaking the form of graphical objects to be provided, which opens thedoor to many applications requiring the distinction to be made betweenmany graphical objects that are presented simultaneously to a user.

The technique is thus adaptable for example to a keyboard display ofalphanumeric characters with a view to identifying the alphanumericcharacter on which the user is attending, or more generally to a displayof a plurality of logos, menus, graphical elements, etc. The modulationproduced by the modulation signal does not affect viewing comfortprovided that the frequency components of this modulation signal arelower than about 25 Hz. A modulation signal for example has a periodicpattern, repeating with a frequency comprised between 2 and 20 Hz, thismodulation frequency being sampled at a sampling frequency correspondingto the refresh rate (which in general is higher than 60 Hz) of themonitor used to display the stimuli.

The reconstruction and search for statistical dependence may be carriedout in real-time, this allowing the graphical object to which theindividual is attending to be identified in real time. In particular, noprior classification of the EEG signals into classes respectivelycorresponding to the various stimuli is required to reconstruct themodulation signal.

In or more embodiments of the method according to the first aspect, theset of at least one visual stimulus comprises a plurality of visualstimuli and the set of at least one modulation signal comprises aplurality of modulation signals and the method furthermore comprisessearching, among the plurality of modulation signals, for a modulationsignal which maximizes a degree of statistical dependence with thereconstructed modulation signal; and identifying the visual stimuluscorresponding to the modulation signal for which the degree ofstatistical dependence is maximal; the modulation signals being composedso that an overall degree of statistical dependence, which is determinedin the time and/or frequency domain, for all the pairs of modulationsignals corresponding to two separate visual stimuli, is lower than asecond threshold.

In one more embodiments of the method according to the first aspect, thereconstruction is carried out by applying a reconstruction model to theplurality of electroencephalographic signals.

The reconstruction model establishes the mathematical relationship thatexists between a modulation signal used to generate a visual stimulusand the electroencephalographic response of the individual focusing hisattention on this visual stimulus. The reconstruction model thus servesto extract from the EEG signals information relevant to a type ofstimulus. This relationship is mainly dependent on the position on theskull of an individual (also referred to as the user) of the electrodesof the piece of equipment used to acquire the EEG signals. Specifically,the visual characteristics of the stimuli (type, size, position, color,etc.) have little impact on this relationship.

The reconstruction model may use various, linear or non-linear,mathematical models allowing the EEG signals to be combined, toreconstruct a modulation signal. Because a modulation signal isreconstructed, and not the visual stimulus as such (i.e. the animatedgraphical object that is displayed on a screen), reconstruction ispossible simply and with precision, for example via simple linearcombination of the EEG signals, and this without restriction on thenature or semantic content of the animated graphical object used asvisual stimulus.

The search for statistical dependence between the reconstructedmodulation signal and the one or more various modulation signals is alsofacilitated. Furthermore, the one or more visual stimuli are displayableon any currently available screen, such as: a computer screen, a tabletscreen, a screen of a phone terminal, etc. It is therefore not necessaryto provide a dedicated system for producing stimuli. Thus, the use ofmodulation signals to generate visual stimuli not only allows thereconstruction and search for statistical dependence to be carried outeasily, but also makes the method flexible and adaptable to any type ofvisual stimuli taking the form of an animated graphical object.

In one or more embodiments of the method according to the first aspect,the reconstruction model comprises a plurality of parameters ofcombination of electroencephalographic signals and the method comprisesdetermining values of the parameters of the plurality of parameters ofcombination of electroencephalographic signals in an initial learningphase.

In one or more embodiments of the method according to the first aspect,the method furthermore comprises, in the initial learning phase appliedto a subset of at least one visual stimulus among the plurality ofvisual stimuli, obtaining, for each visual stimulus of said subset of atleast one visual stimulus, test electroencephalographic signals producedby the individual focusing his attention on the visual stimulus inquestion; and determining optimal values for the plurality of parametersof combination of electroencephalographic signals, for which values theapplication of the reconstruction model to the plurality of testelectroencephalographic signals recorded for a visual stimulus allows areconstructed modulation signal to be generated that approximates asbest as possible the modulation signal corresponding to the visualstimulus in question.

Thus, by determining the parameters of combination of the EEG signalsfor a given acquisition device, it is possible to reliably generate areconstructed modulation signal and to compare it to those used togenerate the visual stimuli. Since this relationship is stable overtime, and not very dependent on the visual stimuli and on the graphicalobjects, these parameters of combination of the EEG signals are reusablefor all the subsequent visual stimuli liable to be presented to anindividual, even if these stimuli are different from those used to, inthe learning phase, determine the reconstruction model.

The reconstruction model lastly allows potential variability from oneindividual to the next to be managed in that the parameters of thereconstruction model may be adjusted for each individual.

According to one or more embodiments, an elementary transformation is atransformation of the set of transformations consisting of a variationin light intensity, a variation in contrast, a colorimetrictransformation, a geometric deformation, a rotation, an oscillation, amovement along a path, a change in shape and a change in graphicalobject or a combination of transformations chosen from said set oftransformations.

The subject of the present description, according to a second aspect, isa computer program containing program-code instructions for executingthe steps of a method according to the first aspect, when said computerprogram is executed by a data processor.

The subject of the present description, according to a third aspect, isa computational system comprising at least one memory for storing codeinstructions of a computer program, configured to execute a methodaccording to the first aspect and at least one data processor configuredto execute such a computer program.

The subject of the present description, according to a fourth aspect, isa system for determining the focus of the visual attention of anindividual from electroencephalographic signals, the system comprisingmeans for implementing the method according to the first aspect,according to any one of the described embodiments.

The system especially comprises a device for generating display signals,which is configured to generate a set of at least one visual stimulus tobe displayed from at least one graphical object, from at least oneelementary transformation and from a set of at least one modulationsignal, a visual stimulus being an animated graphical object obtained byapplying to a graphical object a temporal succession of elementarytransformations that is temporally parameterized by a correspondingmodulation signal; a signal-processing device configured to obtain aplurality of electroencephalographic signals produced by the individual;obtain a reconstructed modulation signal by reconstructing a modulationsignal from the plurality of electroencephalographic signals; compute adegree of statistical dependence between the reconstructed modulationsignal and each modulation signal of said set of at least one modulationsignal; and identify at least one visual stimulus corresponding to amodulation signal for which the degree of statistical dependence ishigher than a first threshold.

According to one or more embodiments of the system according to thefourth aspect, the set of at least one visual stimulus comprises aplurality of visual stimuli and the set of at least one modulationsignal comprises a plurality of modulation signals and thesignal-processing device is furthermore configured to search, among theplurality of modulation signals, the modulation signal for which thedegree of statistical dependence with the reconstructed modulationsignal is maximal; identify the visual stimulus corresponding to themodulation signal for which the degree of statistical dependence ismaximal; the modulation signals being composed so that an overall degreeof statistical dependence, which is determined in the time and/orfrequency domain, for all the pairs of modulation signals correspondingto two separate visual stimuli, is lower than a second threshold.

BRIEF DESCRIPTION OF THE FIGS

Other advantages and features of the technique presented above willbecome apparent on reading the detailed description given below, whichmakes reference to the FIGS, in which:

FIG. 1A schematically shows a system for determining the focus of thevisual attention of an individual from EEG signals according to oneexample of an embodiment;

FIG. 1B schematically shows a computational device according to oneexample of an embodiment;

FIG. 2A schematically shows the data and signals exploited in a systemand method for determining the focus of visual attention according toone example of an embodiment;

FIGS. 2B-2E each schematically show examples of modulation signalsusable in a method or system for determining the focus of visualattention;

FIG. 3 schematically shows aspects of a method and system fordetermining the focus of visual attention;

FIG. 4A is a flowchart of a method for generating an EEG signalreconstruction model according to one example of an embodiment;

FIG. 4B is a flowchart of a method for determining the focus of thevisual attention of an individual according to one example of anembodiment;

FIG. 5 illustrates an example of an animated graphical object;

FIG. 6 shows examples of visual stimuli;

FIG. 7 illustrates an example of application of a system and method fordetermining the focus of visual attention.

In the various embodiments that are described with reference to theFIGS, elements that arc similar or identical have been referenced withthe same references.

DETAILED DESCRIPTION

The present description is given with reference to functions, functionalunits, entities, block diagrams and flowcharts that describe variousembodiments of methods, systems and programs. Each function, functionalunit, entity and step of a flowchart may be implemented by software,hardware, firmware, microcode or any appropriate combination of thesetechnologies. When software is used, the functions, functional units,entities or steps may be implemented by computer-program instructions orsoftware code. These instructions may be stored or transmitted to acomputer-readable storage medium and/or be executed by a computer inorder to implement these functions, functional units, entities or steps.

The various embodiments and aspects described below may be combined orsimplified in multiple ways. In particular, the steps of the variousmethods may be repeated for each set of graphical objects in questionand/or each user in question, the steps may be inverted, executed inparallel and/or executed by various computational entities. Only certainembodiments of examples are described in detail in order to ensure theclarity of the description but these examples are not intended to limitthe general scope of the principles that this description considered inits entirety should make clear.

FIG. 1A schematically shows an example of an embodiment of a system 100for determining the focus of the visual attention of an individual (alsoreferred to as the user below) 101 from electroencephalographic signals.

In one or more embodiments, the system 100 comprises a display screen105 configured to display animated graphical objects, a device 110 forgenerating display signals, a signal-processing device 120, a piece ofequipment 130 for acquiring EEG signals and a device 140 for controllingthe piece of equipment 130.

In one or more embodiments, the device 110 for generating displaysignals is configured to generate display signals to be displayed by thedisplay screen 105. These display signals encode a plurality of visualstimuli intended to be presented to the user 101 by means of the displayscreen 105.

In one or more embodiments, the piece of equipment 130 is configured toacquire EEG signals. This piece of equipment for example takes the formof a headcap equipped with electrodes intended to make contact with theskull of the user 101. Such a headcap is for example a headcapmanufactured by Biosemi®, which is equipped with 64 electrodes. Othertypes of equipment are usable: for example the Geodesic™ EEG devicessold by Electrical Geodesics Inc. (EGI)®, or those sold by CompumedicsNeuroScan®, which usually count between 16 and 256 electrodes. In therest of the description, it is assumed, by way of example, that thepiece of equipment 130 takes the form of a headcap.

In one or more embodiments, the signal-processing device 120 isconfigured to process the EEG signals acquired by means of the headcap130 for acquiring EEG signals.

In one or more embodiments, the device 140 for controlling the piece ofequipment 130 is a device that serves as an interface between theheadcap 130 and/or the signal-processing device 120. The control device140 is configured to control the acquisition of EEG signals and toobtain the EEG signals acquired by the headcap 130. In particular, thedevice 140 for controlling the headcap 130 is configured to send acommand to trigger the acquisition of the EEG signals.

All or some of the functions described here with respect to the device110 for generating display signals, the signal-processing device 120 andthe control device 140 may be carried out by software and/or hardwareand implemented in at least one computational device comprising a dataprocessor and at least one memory for storing data.

In one or more embodiments, each device 110, 120, 140 and each of thesteps of the described methods are implemented by one or more physicallyseparate computational devices. Conversely, the various devices 110,120, 140 may be integrated into one and the same computational device.Likewise, the steps of the methods described here may be implemented byone and the same computational device. The piece of equipment 130 foracquiring EEG signals may also comprise a computational device and beconfigured (data processor, memory, etc.) to implement all or some ofthe steps of the methods described in this document.

Each computational device has on the whole the architecture of acomputer, including the components of such an architecture: one or moredata memories, one or more processors, communication buses, one or moreuser interfaces, one or more hardware interfaces for the connection ofthis computational device to a network or another piece of equipment,etc.

An example of an embodiment of such an architecture is illustrated inFIG. 1B. This architecture comprises a processing unit 180 including atleast one data processor, at least one memory 181, one or more datastorage media 182, and hardware interfaces 183 such as networkinterfaces and interfaces for the connection of peripherals, and atleast one user interface 184 including one or more input/output devicessuch as a mouse, keyboard, display, etc. The data storage medium 182comprises code instructions of a computer program 186. Such a storagemedium 182 may be an optical storage device such as a compact disc (CD,CD-R or CD-RW), DVD (DVD-ROM or DVD-RW) or Blu-ray disc, a magneticmedium such as a hard disk, a flash memory, a magnetic tape or floppydisk, a removable storage medium such as a USB key, an SD or micro-SDmemory card, etc. The memory 181 may be a random-access memory (RAM), aread-only memory (ROM), a cache memory, a non-volatile memory, a back-upmemory (for example flash or programmable memories), read-only memoriesor any combination of these types of memory. The processing unit 180 maybe any microprocessor, integrated circuit or central processing unitcomprising at least one processor based on computational hardware.

FIG. 2A schematically shows the data and signals exploited in a systemand method for determining the focus of visual attention. Thecorresponding notations will now be introduced.

In one or more embodiments, a plurality of graphical objects O1, O2, . .. , ON intended to be presented to a user 101 is used. Each of thesegraphical objects may be an alphanumeric character (number or letter orother character), a logo, an image, a text, an element of auser-interface menu, a user-interface button, an avatar, a 3-D object,etc. Each of these graphical objects may be coded via a bitmap or vectorimage.

In one or more embodiments, one or more elementary transformations T1,T2, . . . , TP are defined in order to be applied to the graphicalobjects O1, O2, . . . , ON. An elementary transformation may be: avariation in light intensity, a variation in contrast, a colorimetrictransformation, a geometric deformation, a rotation, an oscillation, amovement along a planar or three-dimensional path, a change of shape, oreven a change of graphical object, etc. A change of graphical object mayfor example correspond to a transformation that replaces a graphicalobject with another graphical object of the same category, for example,the replacement of a letter (e.g. A) with another letter (e.g. B), anumber with another number, a logo with another logo, etc. An elementarytransformation may also be a combination of a plurality of theaforementioned elementary transformations.

Each of these elementary transformations is parameterizable by at leastone parameter of application.

In one or more embodiments, a parameter of application defines a degreeof transformation of the elementary transformation on a preset scale. Ascale from 0 to 100 or from −100 to +100 may for example be used.

For example, when the elementary transformation is a variation in lightintensity, this variation in intensity may be applied with a degree oftransformation variable between 0 and 100, a degree of transformationequal to 0 meaning that the image coding the graphical object is notmodified, and a degree of transformation equal to 100 indicating thatthe image becomes completely white or, in contrast, black. By making thedegree of transformation vary between 0 and 100, an effect is obtainedwhereby the image appears to flash.

According to another example, when the elementary transformation is avariation in contrast, this variation in contrast may be applied with adegree of transformation variable between 0 and 100, a degree oftransformation equal to 0 meaning that the image coding the graphicalobject is not modified, a degree of transformation equal to 100indicating that the contrast of the image becomes maximal (the imagebecomes a black-and-white image, if it is coded in greyscale).

Likewise, for a geometric deformation of morphing type, a degree oftransformation may correspond to the degree of morphing. For a rotation,a degree of transformation may corresponding to an angle of rotation.For an oscillation, a degree of transformation may correspond to a rateand/or amplitude of oscillation. For a movement on a path, a degree oftransformation may correspond to a distance travelled and/or a speed ofmovement on the path. For a change of shape (of object category,respectively), a degree of transformation may correspond to a rateand/or amplitude reflecting the passage from one shape (category,respectively) to the other.

For each of the graphical objects O1, O2, . . . , ON, a correspondingmodulation signal SM1, SM2, SMN is generated. A modulation signal servesto define the variations as a function of time in one or more parametersof application of the elementary transformation applied to the graphicalobject in question. For example, the degree of transformation di(t) atthe time t is defined by the amplitude SMi(t) of the modulation signalSMi at the time t.

In one or more embodiments, an animated graphical object OA1, OA2, . . ., OAN is generated for each corresponding graphical object O1, O2, . . ., ON from one or more corresponding elementary transformations and froma corresponding modulation signal. The animated graphical object OA1,OA2, OAN thus generated is presented on a display screen 105.

In one or more embodiments, a visual stimulus is an animated graphicalobject OAi (i integer number comprised between 1 and N) obtained byapplying to a corresponding graphical object Oi a temporal successionSTi of elementary transformations that is temporally parameterized by acorresponding modulation signal SMi. Thus, at each time tz of a discretesequence of times t0, t1, . . . tz, . . . in a time interval [tmin,tmax], a modified graphical object OAi(tz) is generated, by applying acorresponding elementary transformation Ti to the graphical object Oiwith a degree of transformation di(tz) corresponding to the amplitudeSMi(tz) at the time tz of the corresponding modulation signal SMi to thegraphical object Oi. The animated graphical object thus corresponds tothe temporal succession of the modified graphical objects OAi(tz) whentz varies in the time interval [tmin, tmax].

In one or more embodiments, EEG signals, denoted E1, E2, . . . , EX, areacquired by means of a piece of equipment 130 for acquiring EEG signals.From the EEG signals E1, E2, . . . , EX, a reconstructed modulationsignal SMR is generated.

In one or more embodiments, the reconstructed modulation signal SMR isgenerated by applying a reconstruction model MR to the signals E1, E2, .. . , EX. The parameters of the reconstruction model MR are denoted P1,P2, . . . PK.

The reconstruction model MR may be a linear model, the reconstructedmodulation signal being a linear combination of the signals E1, E2, . .. , EX.

Other more elaborate models may be used, in particular models based onneural networks.

In one or more embodiments, a modulation signal is composed ofelementary signals. These elementary signals may be square-wave signals,triangle-wave signals, sinusoidal signals, etc. The modulation signalsmay have different durations. In one or more embodiments, the modulationsignals are periodic, a temporal pattern being periodically reproducedby each modulation signal. A modulation signal for example has aperiodic temporal pattern, repeating with a frequency comprised between2 and 20 Hz, this modulation signal being sampled at a samplingfrequency corresponding to the refresh frequency (which in general ishigher than 60 Hz) of the screen on which the visual stimuli (i.e. theanimated graphical objects) generated from the modulation signals aredisplayed.

The amplitude of the modulation signal SMi serves to define a degree oftransformation. The relationship between the amplitude of the modulationsignal SMi and the degree of transformation may or may not be linear.The amplitude of a modulation signal SMi may vary between a minimumvalue (corresponding to a 1st° of transformation) and a maximum value(corresponding to a 2nd° of transformation).

In one or more embodiments, the modulation signals are in this caseindependent pairwise: the modulation signals are composed so that thedependence, measured in the time and/or frequency domain, between anytwo separate modulation signals is minimal (for example zero) or lowerthan a given threshold SC1. The dependence between two signals may bequantified by a degree of statistical dependence. The degree ofstatistical dependence between two modulation signals may be computed,in the time domain, for example via a coefficient of temporalcorrelation and/or, in the frequency domain, for example via the degreeof spectral coherence.

In one or more embodiments, for each pair of modulation signalscorresponding to separate visual stimuli, the modulation signals arctemporally decorrelated pairwise.

In one or more embodiments, a degree of statistical dependence may becomputed for each pair of modulation signals corresponding to separatevisual stimuli, then an overall degree of statistical dependence (forexample, the average degree of dependence, the maximum degree ofdependence or the cumulative degree of dependence) may be computed forall the pairs of modulation signals. The modulation signals aredetermined by searching for the modulation signals that minimize thisoverall degree of statistical dependence or that allow an overall degreeof statistical dependence that is below a preset threshold SC1 to beobtained.

In one or more embodiments, the degree of statistical dependencecomputed for each pair of modulation signals corresponding to separatevisual stimuli is zero or below a threshold SC1.

In one or more embodiments, the degree of statistical dependence betweentwo modulation signals may be computed to be the coefficient of temporalcorrelation between these signals, where the correlation coefficientρ(X,Y) between the two signals X and Y may be obtained using Pearson'sformula:

${\rho( {X,\ Y} )} = {\frac{\lbrack {( {X - (X)} )( {Y - (Y)} )} \rbrack}{\sigma_{X}\sigma_{Y}} = \frac{\lbrack{XY}\rbrack - {\lbrack X\rbrack\lbrack Y\rbrack}}{\sigma_{X}\sigma_{Y}}}$

where

is the expected value of a signal, and σ its standard deviation. Thecoefficient of temporal correlation is comprised between 0 and 1, thevalue 0 corresponding to temporally decorrelated signals.

The degree of statistical dependence between the modulation signals maybe determined via other mathematical criteria, such as Spearman'scorrelation coefficient or Kendall's correlation coefficient, orreplaced by measurements of statistical dependence such as mutualinformation.

The degree of spectral coherence between two signals x (t) and y (t) isa real-valued function that may be defined for example by the ratio

|Gxy(f)|²/(Gxx(f)*Gyy(f))

where Gxy (f) is the cross spectral density between x and y, and Gxx (f)and Gyy (f) the power spectral density of x and y, respectively.

In one or more embodiments, the degree of statistical dependence iscomputed over a reference period, for example corresponding to theduration of the reconstruction window (see step 414) and/or the decodingwindow (see step 415).

An effective discrimination between the modulation signals is possiblewhen the overall degree of statistical dependence (for example computedto be the average degree of dependence, the maximum degree of dependenceor a cumulative degree of dependence), computed for all the pairs ofmodulation signals corresponding to separate visual stimuli, is zero(for example for temporally decorrelated signals) or lower than athreshold SC1, for example chosen to be equal to 0.2 (i.e. 20%respectively if this degree is expressed in percent). The lower theoverall degree of statistical dependence, the easier and more effectivethe identification of the visual stimulus observed by an individualattending this stimulus will be. The probability of error indiscrimination (corresponding to the percentage of cases in which thevisual stimulus identified in step 415 is not that to which theindividual is actually attending), among the set of modulation signals,of the modulation signal that will serve to generate the visual stimulusobserved by the subject, is also correspondingly lower. The thresholdSC1 may depend on the choice of the type of modulation signals. Inpractice, it is possible to set a maximum probability of discriminationerror (a probability acceptable for a given application for example),and to adjust the modulation signals so as to remain below this maximumdiscrimination error rate. It will be understood here that, even in thecase where the reconstruction (see step 414) is ideal (i.e. thereconstructed signal is equal at each time to one of the modulationsignals SMi), the quality of the decoding (see step 415) depends on thedegree of statistical dependence between the modulation signals, becauseif the modulation signals SMi are entirely dependent, it will beimpossible to select one modulation signal SMi rather than another.

Each of FIGS. 2B, 2C, 2D and 2E schematically shows a set of modulationsignals usable in a system for determining the focus of visual attentionin order to generate visual stimuli.

FIG. 2B shows a first exemplary set of 10 modulation signals, signal 1to signal 10, that are temporally decorrelated pairwise. These 10signals are periodic sinusoidal signals having different frequencies(and therefore different periods) varying, in steps of 0.2 Hz, between 1Hz and 2 Hz, such that any two signals in this first set do not have thesame frequency. The phases of these signals are unimportant. In theexample shown in FIG. 2B, the amplitude of these signals varies between0% and 100%, meaning that the corresponding degree of transformationvaries (with a suitable coefficient of proportionality) between aminimum value and a maximum value. For all the pairs of modulationsignals of this set of 10 signals, the degree of spectral overlap iszero (i.e. absence of common frequency components) and the maximumcoefficient of temporal correlation in all the pairs of modulationsignals is 0.2, this coefficient of temporal correlation being computedin a correlation window of 4 seconds.

FIG. 2C shows a second exemplary set of 10 modulation signals, signal 1to signal 10, that are temporally decorrelated pairwise. These 10signals are periodic square-wave signals having different frequencies(and therefore different periods) varying, in steps of 0.2 Hz, between 1Hz and 2 Hz, such that any two signals in this second set do not havethe same frequency. The phases of these signals are unimportant. As inFIG. 2B, the amplitude of these signals varies between 0% and 100%meaning that the corresponding degree of transformation varies (with asuitable coefficient of proportionality) between a minimum value and amaximum value. For all the pairs of modulation signals of this set of 10signals, the degree of spectral overlap may be nonzero if certainharmonic components are common but the maximum coefficient of temporalcorrelation in all the pairs of modulation signals is 0.17, thiscoefficient of temporal correlation being computed in a correlationwindow of 4 seconds.

FIG. 2D shows a third exemplary set of 10 modulation signals, signal 1to signal 10, that are temporally decorrelated pairwise. These 10signals are periodic signals having the same period (called thereference period in FIG. 2D) and are composed of elementary square-wavesignals such that the temporal patterns of any two signals in this thirdset are different in the reference period. In this case, the phase ofeach of the signals is important in that it must be adjusted so as tolimit the coefficient of temporal correlation, and therefore the degreeof statistical dependence, to a maximum value for each pair of separatemodulation signals that is selected from this set of 10 modulationsignals. As in FIG. 2B, the amplitude of these signals varies between 0%and 100% meaning that the corresponding degree of transformation varies(with a suitable coefficient of proportionality) between a minimum valueand a maximum value.

FIG. 2E shows a third exemplary set of 9 modulation signals, signal 1 tosignal 9, that are temporally decorrelated pairwise. These signals areperiodic signals having the same period (called the reference period inFIG. 2E). Each of the modulation signals comprises a temporal patterncomposed of a short square-wave pulse of 100% amplitude followed by asignal of longer duration of 0% amplitude, the square-wave pulses of thevarious visual modulation signals being offset in time with respect tothe others so that, at a given time, one single modulation signal has anamplitude of 100% whereas the others have an amplitude of 0%. Thesemodulation signals all have the same temporal pattern (with a differentphase shift) and it is by adjusting the phase of each of the signalsthat it is possible to control the coefficient of temporal correlation,and therefore the degree of statistical dependence, between two signals.When the transformation function used is a function that changesbrightness, the graphical object being visible (unchanged brightness),when the modulation signal is at 100% and invisible when the modulationsignal is at 0% (zero brightness), the obtained animated graphicalobjects, which are obtained from these modulation signals and from thiselementary transformation, flash by appearing and disappearing in agiven order, a single visual stimulus being visible at a given time. Forall the pairs of modulation signals of this set of 10 signals, thecoefficient of temporal correlation is zero.

It is therefore possible to obtain temporally decorrelated modulationsignals using different frequencies, phases or temporal patterns foreach pair of modulation signals, for example:

-   -   With signals composed of the same periodic temporal pattern, but        having different frequencies and therefore different periods        (case of FIGS. 2B and 2C), phase being unimportant;    -   With signals composed of periodic different temporal patterns        that optionally have the same duration (i.e. the same signal        period), with specific phases specific to each temporal pattern        (case of FIG. 2D);    -   With signals composed of the same periodic temporal pattern        having the same period, but the patterns being phase shifted        with respect to one another (case of FIG. 2E).

FIG. 3 schematically shows aspects of a method and system fordetermining the focus of visual attention.

In one or more embodiments, the reconstructed modulation signal SMR iscompared to each of the modulation signals SM1, SM2, . . . , SMN inorder to seek a modulation signal for which the degree of statisticaldependence is maximal. For example, if the degree of statisticaldependence is maximal for the modulation signal SM4, this means that thevisual attention of an individual is focused on the visual stimulus OA4generated from this modulation signal SM4.

In the example shown in FIG. 3 , the visual stimuli are the numbers 0 to9, the visual attention of the individual is focused on the number 4corresponding to visual stimulus OA4. The maximum degree of statisticaldependence is found with the corresponding modulation signal SM4.

An example of an embodiment of a method for generating a reconstructionmodel MR is schematically illustrated in FIG. 4A. Although the steps ofthis method are presented sequentially, certain at least of the stepsmay be omitted or indeed be executed in a different order or indeed beexecuted in parallel or even combined in order to form only a singlestep.

In a step 401, a trial i (i∈[1; N]) is carried out with a visualstimulus generated from a modulation signal SMi: the visual stimulus ispresented on a display screen and an individual is invited to observethe visual stimulus, i.e. to turn his visual attention to this visualstimulus. Each visual stimulus is an animated graphical object obtainedby applying, to a graphical object, a temporal succession of elementarytransformations that is temporally parameterized by a correspondingmodulation signal. Test EEG signals Ei,j are recorded while theindividual is attending the visual stimulus in question, where i is theindex identifying the trial and the corresponding modulation signal, jis the index identifying the EEG channel recorded. Each of these EEGsignals Ei,j is composed of a plurality of EEG segments Ei,j,k, where kis the index identifying the segment.

In an example of implementation of step 401, ten visual stimuli takingthe form of flashing numbers (numbers ranging from 0 to 9) are displayedon a screen, each flashing at a slightly different frequency. Theindividual is equipped with an EEG headcap and views a display screen onwhich the ten numbers flash at different frequencies. A succession oftrials is carried out. Each trial lasts for example about ten seconds,the interval between two trials for example being 1 or 2 seconds. Ineach trial, the individual is instructed to attend one of the numbersand must ignore the others until the following trial. The individualthus switches his attention from one stimulus to the next and generatesEEG signals E1, E2, . . . , EX at different frequencies depending on thefocus of his visual attention.

In a first variant embodiment, the EEG segments are time stamped in step401. The modulation signals are also time stamped in step 401. The timestamping may be carried out using any method.

In this first variant embodiment, the clock of the piece of acquiringequipment 130 is used to timestamp the EEG segments and to produce atime code t_(i)′ for each EEG segment. The clock of the control device140 is used to timestamp the modulation signals and to produce a timecode each time a preset event occurs in the stimulation (for examplecorresponding to the appearance of a new visual stimulus on the screenor to the start of the display of a stimulus).

In a second variant embodiment, an additional EEG channel is used, viawhich short electrical pulses of known amplitude are transmitted eachtime a preset event occurs in the simulation (for example correspondingto the appearance of a new visual stimulus on the screen or to the startof the display of a stimulus). This additional EEG channel containingthe short pulses is saved with the segments of EEG data.

Step 401 is repeated a plurality of times, for each visual stimulus of aplurality of visual stimuli, so as to record the corresponding EEGsignals produced by the individual when he is attending the visualstimulus in question.

In a step 402, the EEG segments Ei,j,k and the modulation signals SMiare temporally aligned (or synchronized). This synchronization may becarried out using any method.

This synchronization may use the double timestamp of the EEG segmentsand of the modulation signals, or indeed the additional EEG channel.

When the double timestamp is used, and the time codes are produced usingtwo different clocks, it is necessary to correct these values so as toobtain timestamps produced virtually by the same reference clock, so asto correct for potential temporal drift between the clocks. For example,when the clock of the control device is used as reference clock, thetime codes t1′ of the time stamped EEG segments Ei,j,k produced by theclock of the acquiring device are resynchronized with respect to thereference clock to obtain time codes t1. By associating these correctedtime codes of the EEG segments with those produced for the modulationsignals, it is possible to achieve the alignment between the EEGsegments Ei,j,k and the signals SMi.

The difference between the reference clock (1) of the control device 140and that of the piece of equipment 130 for acquiring the EEG data (t′)is modelled by a linear equation: diff=a*(t′−t₀)+b=t′−t, where a is thedrift between the two clocks and b is the offset at t′=t₀. To estimatethese coefficients a and b, a series of x points (t′, diff(t′)) areacquired, prior to step 401, then the coefficients a and b are estimatedusing the least-squares method. In order to compensate for randomvariations in the time taken to execute the instructions and to transmitdata between the control device 140 and the piece of acquiring equipment130, each point (t′, diff(t′) is obtained by the control device 140successively sending n time codes t_(k) to the piece of acquiringequipment 130, which produces, each time a time code t_(k) is received,a time code t_(k)′. The point (t′, diff(t′)) retained for thecomputation of the coefficients a and b corresponds to the pair (t_(k)′,diff=t_(k)′−t_(k)) for which the difference (t_(k)′−t_(k)) is minimal.Once the coefficients a and b have been obtained, the time codes t_(i)′are corrected in the following way:

t _(i) =t _(i) ′−a*(t _(i) ′−t ₀)−b

The time stamping and synchronizing steps are however optional and arein particular not necessary when the piece of acquiring equipment 130and the control device use the same clock.

In a step 403, the EEG segments Ei,j,k are concatenated so as togenerate the EEG signals Ei,j.

In a step 404, preprocessing and denoising may be applied to the EEGsignals so as to optimize the signal/noise ratio. Specifically, the EEGsignals may be considerably contaminated by artefacts both of intra- andextra-cerebral origin, for example electrical artefacts (such as at 50Hz, frequency of the current of the mains grid in Europe) or biologicalartefacts (such as eye movements, the electrocardiogram, muscularactivity, etc.). In one or more embodiments, the signals E1, E2, . . . ,EX are thus denoised prior to the generation of the reconstructedmodulation signal SMR. This denoising may consist in simply filteringhigh frequencies from the signals E1, E2, . . . , EX, for example allthe frequencies higher than 40 Hz in order to remove the electricalnoise produced by the mains grid. Multivariate statistical approachesmay be used, in particular principal component analysis (PCA),independent component analysis (ICA) and canonical correlation analysis(CCA), allowing the useful components of the EEG signal (i.e. those dueto the brain activity related to the cognitive task being carried out)to be separated from the irrelevant components.

In a step 405, the parameters of the reconstruction model arcdetermined. This determination may be carried out so as to minimize thereconstruction error. The reconstruction model for example comprises aplurality of parameters of combination of EEG signals. These parametersof combination are determined using a method of solving mathematicalequations so as to determine optimal values for the parameters ofcombination, i.e. the values for which the application of thereconstruction model to the plurality of test EEG signals Ei,j recordedfor a visual stimulus allows a reconstructed modulation signal to begenerated that approximates as best as possible the modulation signalcorresponding to the visual stimulus in question, i.e. the values forwhich the reconstruction error is minimal.

In one or more embodiments, the values α_(j) of the parameters ofcombination may be fixed (independent of time). In other embodiments,these values may be adjusted in real-time in order to take into accounta potential adaptation of the brain activity of the user 101, or avariation in the signal/noise ratio in the EEG signal during a recordingsession.

The reconstruction model MR may be a linear model that produces amodulation signal via linear combination of the signals Ei,j. In thiscase, the parameters of combination are parameters α_(ij) of linearcombination and the mathematical equations are linear equations of theform:

SMi=Σ _(j)α_(j) Ei,j for i∈[1;N]

Other more elaborate models may be used, in particular models based onneural networks, with which the modulation signal is obtained byapplying, in cascade, non-linear mathematical operations to the signalsEi,j. For example, a Siamese network, in which a neural network istrained (on the basis of calibration data) to make any EEG signal Ecorrespond to a one-dimensional time signal R (in the present case, amodulation signal) so that, two EEG signals E1 and E2 recorded atdifferent times respectively produce one-dimensional signals R1 and R2(in the present case, modulation signals) that are similar when theattention of the individual is focused on the same animated graphicalobject, and dissimilar when the attention of the individual is focusedon two separate animated graphical objects. The notion of similaritybetween two signals is defined in the mathematical sense (it may forexample be a question of a simple correlation) and corresponds to afunction that quantifies the degree of similarity between two objects(see for example the page:https://en.wikipedia.org/wiki/Similarity_measure). A plurality ofmathematical definitions of similarity may be used, such as for examplethe inverse of the Euclidean distance, or even the “cosine similarity”(see for example the page: https://en.wikipedia.org/wiki/Cosinesimilarity).

The reconstructed modulation signal is a one-dimensional signal Rgenerated by the neural network from a newly acquired EEG sample E.

In one or more embodiments, the steps of the method for generating areconstruction model are implemented by a system 100 according to FIG.1A, for example by the signal-processing device 120.

An example of an embodiment of a method for determining the focus of thevisual attention of an individual is schematically illustrated in FIG.4B. Although the steps of this method are presented sequentially,certain at least of these steps may be omitted or indeed be executed ina different order or indeed be executed in parallel or even combined toform only a single step.

In one or more embodiments, the steps of the method for determining thefocus of visual attention are implemented by a system 100 according toFIG. 1A, for example by the signal-processing device 120 and the device110 for generating display signals.

In a step 411, the device 110 for generating display signals isconfigured to generate a plurality of visual stimuli from a plurality ofgraphical objects O1, O2, . . . , ON, from a plurality of elementarytransformations T1, T2, . . . , TP and from a plurality of modulationsignals SM1, SM2, SMN. A visual stimulus is an animated graphical objectOAi (i comprised between 1 and N) obtained by applying to acorresponding graphical object Oi a temporal sequence STi of elementarytransformations that is temporally parameterized by a correspondingmodulation signal SMi.

In one or more embodiments, the number N of visual stimuli, ofmodulation signals and of graphical objects is equal to 1.

In one or more embodiments, the number P of elementary transformationsis equal to 1. Each elementary transformation of the temporal sequenceSTi of elementary transformations may thus correspond to a givenelementary transformation a parameter of application of which variesover time.

Over time, the individual may pass his visual attention from oneanimated graphical object to another. During this time, in a step 412,the electroencephalographic signals E1, E2, . . . , Ej, . . . , EXproduced by the individual are recorded by the piece of acquiringequipment 130.

In one or more embodiments, the signal-processing device 120 isconfigured to obtain a plurality of electroencephalographic signals E1,E2, . . . , Ej, . . . , EX produced by the individual focusing hisattention on one of the visual stimuli OAi.

In a step 413, the electroencephalographic signals E1, E2, . . . Ej, . .. EX arc preprocessed and denoised so as to improve the reliability ofthe method for determining the focus of visual attention. Thepreprocessing may consist in synchronizing the segments ofelectroencephalographic signals E1, E2, . . . Ej, . . . EX with respectto a reference clock, as explained above with respect to step 402, inconcatenating the segments of electroencephalographic signals asexplained above with respect to step 403, and/or in denoising theelectroencephalographic signals as explained above with respect to step404.

In one or more embodiments, the signal-processing device 120 isconfigured, in a step 414, to obtain a reconstructed modulation signalSMR by reconstructing a modulation signal from a plurality ofelectroencephalographic signals E1, E2, . . . , Ej, . . . , EX.

In one or more embodiments, the signal-processing device 120 isconfigured to reconstruct a modulation signal and to generate areconstructed modulation signal SMR from the plurality ofelectroencephalographic signals obtained in step 413 or 412 (with orwithout preprocessing and/or denoising). In one or more embodiments, thereconstruction is carried out by applying a reconstruction model to theplurality of electroencephalographic signals obtained in step 413 or 412(with or without preprocessing and/or denoising). This reconstructionmay be carried out in a given moving time window, here called thereconstruction window, and periodically repeated for each temporalposition of the reconstruction window.

For example, when the reconstruction model MR is a linear model thatproduces a modulation signal via a linear combination of the signals E1,E2, . . . , Ej, . . . , EX. In this case, the parameters of combinationare the parameters α_(j) of linear combination obtained in step 405 andthe reconstructed modulation signal SMR is computed via linearcombination of the signals E1, E2, . . . , Ej, . . . , EX:

SMR=Σ _(j)α_(j) Ej

In one or more embodiments, the signal-processing device 120 isconfigured, in a step 415 (called the decoding step), to compute adegree of statistical dependence between the reconstructed modulationsignal and each modulation signal of the set of modulation signals andto identify at least one visual stimulus corresponding to a modulationsignal for which the degree of statistical dependence is higher than athreshold SC2, of value for example comprised between 0.2 and 0.3. Thefact of identifying at least one visual stimulus corresponding to amodulation signal for which the degree of statistical dependence ishigher than a threshold SC2 means that the visual attention of theindividual is being given, a priori, to this visual stimulus and/or thatthese one or more visual stimulus have just appeared in a zone of thedisplay screen observed by the individual. It is therefore possible touse this identification to detect that a change in display has occurredand/or that a change in focus of visual attention has occurred. Thedegree of statistical dependence may be determined as described above inthis document. The degree of statistical dependence is for example acoefficient of temporal correlation between the reconstructed modulationsignal and a modulation signal of the set of modulation signals.

In one or more embodiments, the number N of visual stimuli, ofmodulation signals and of graphical objects is strictly higher than 1and the signal-processing device 120 is furthermore configured, in astep 415, to seek, among the plurality of modulation signals SM1, SM2, .. . , SMN, the modulation signal SMi for which the degree of statisticaldependence with the reconstructed modulation signal SMR is maximal andto identify the visual stimulus OAi corresponding to the modulationsignal SMi for which the degree of statistical dependence is maximal.The visual attention is a priori focused on the identified visualstimulus OAi. The search is for example carried out by computing adegree of statistical dependence between the reconstructed modulationsignal SMR and each signal of the plurality of modulation signals SM1,SM2, . . . , SMN. This decoding step may be carried out in a givenmoving time window, here called the decoding window, and periodicallyrepeated for each temporal position of the decoding window. The durationof the decoding window may be identical to that of the reconstructionwindow.

In one or more embodiments, when the number N of visual stimuli, ofmodulation signals and of graphical objects is strictly higher than 1,one or more visual stimuli may be displayed at a given time on thedisplay screen 105. The decoding step 415 may nevertheless be identicalwhatever the number of visual stimuli displayed at a given time, thestatistical dependence being able to be sought with any one of themodulation signals SM1, SM2, SMN corresponding to the visual stimulicapable of being displayed. Thus, the need to modify dynamically and tosynchronize the processing operations carried out in the decoding step415 with respect to the variations in the content actually displayed isavoided. This may be very useful when the visual stimuli are integratedinto a video or when the user interface, into which the visual stimuliare integrated, is modified dynamically as the user interacts with thisuser interface.

In one example embodiment, 10 visual stimuli taking the form of flashingnumbers (numbers ranging from 0 to 9) are displayed on a screen, eachflashing at a slightly different frequency or with the same frequencybut appearing alternately on the screen. The individual is equipped withan EEG headcap and views a display screen on which the 10 numbers flashat different frequencies.

In one embodiment, during the determination of the focus of visualattention, in case of ambiguity between two or more visual stimuli or incase of perturbations and/or artefacts in the recorded EEG signals due,for example, to movements of the user, it is possible to temporarilymodify (for example, during the time required to remove the ambiguity orto obtain less perturbed signals) the modulation signals of the visualstimuli. This modification may be carried out so as, for example, todisplay only visual stimuli for which there is ambiguity and/or tomodify the modulation signals of the visual stimuli for which there isambiguity.

The modification of the modulation signals may consist in modifying thefrequency or the temporal pattern of the modulation signals, so as toincrease the frequency and/or the total duration of visibility and/orthe degree of transformation of the visual stimuli for which there isambiguity and to decrease the frequency and/or the total duration ofvisibility and/or the degree of transformation of the other visualstimuli. The modification may also consist in permuting the modulationsignals between one another, without changing their temporal pattern ortheir frequency. This permutation may be random. Such a permutationamounts, when the modulation signals and the elementary transformationare defined so that the visual stimuli flash by appearing anddisappearing on the display screen in a given order (see the example ofFIG. 2E), in modifying, for example randomly, the order of appearance ofthe visual stimuli, so that the stimuli for which there is ambiguity arevisible more frequently. The permutation may be combined with amodification of the modulation signals with the aim of increasing thefrequency and/or the total duration of visibility and/or the degree oftransformation of the visual stimuli for which there is ambiguity.

The signal-processing device 120 allows, without any information otherthan the EEG, the visual stimulus on which attention is focused to beautomatically identified. A reconstruction model allows, from the rawEEG, the reconstructed modulation signal to be generated. The modulationsignal is correlated to the modulation signals corresponding to thevarious animated graphical objects, the observed visual stimulus beingthat corresponding to the modulation signal for which the degree ofstatistical dependence is maximal.

Tests obtained by grouping the results of a plurality of individualsallow it to be showed that the method for determining the observedvisual stimulus is very robust (error rate lower than 10% for signalsE1, E2, . . . , EX recorded over a time of 1 second), even when thelearning phase is very short, i.e. a few minutes (for example less than5 minutes) or even a few seconds (for example less than 5 seconds) inlength, and carried out with a few stimuli types.

The method for determining the focus of visual attention is applicablenot merely to numbers but also to many human-machine interfaces, and forexample to a complete alphanumeric keyboard comprising 26 or morecharacters.

FIG. 5 illustrates another example of a human-machine interface to whichthis method is applicable. In this example, the dynamic stimuli consistof logos or icons that are animated, not by elementary transformationsthat act on the light intensity of the logo, but by applying movementsso as to move the logo about itself, in a plane or in athree-dimensional space. These movements are for example oscillations orrotations, at various frequencies decodable in real-time. In this case,the amplitude of the corresponding modulation signal indicates thedegree of transformation at a given time, i.e. the angle of rotation tobe applied at a given time. These movements are for example periodic.

In FIG. 5, 12 logos APP1 to APP12 have been shown, said logos beingarranged in a grid of 4×3 logos. Although this figure has been presentedin black and white, the logos may also be in color. In the example ofFIG. 5 , the logos pivot about themselves at various frequencies, asillustrated in FIG. 5 for the logo APPS. Each of these logos is animatedby a rotational oscillation about itself that occurs at a frequencydifferent from that of the other logos. The periodic rotation applied tothe icons induces cerebral responses that are detectable in the EEGsignals at the specific frequency of rotation, and which may be decodedin real-time by virtue of the techniques described above. This type ofinterface is highly flexible and could in particular be employed in asmart phone or tablet. Such a human-machine interface for example allowsgraphical interfaces to be produced for any type of computationaldevice, for example for software applications and/or operating systemson a mobile terminal or computer, whether the display screen is a touchscreen or not.

FIG. 6 illustrates another example of a usable human-machine interface.In order to facilitate the focus of the visual attention of anindividual on a visual stimulus and to minimize the influence, on theEEG signals, of neighboring visual stimuli, a technique referred to as“crowding” may be called upon. This technique consists in encirclingeach visual stimulus with optionally animated lateral masks thatdecrease the visual perturbations related to the animation ofneighboring visual stimuli and allow the individual to more effectivelyfocus his attention on one stimulus in particular and the decodingthereof therefore to be improved.

FIG. 7 illustrates another example of a human-machine interface in whichfeedback is given to the user on the visual stimulus that has beenidentified as being observed by the user. The human-machine interfacecomprises the numbers 0 to 9. In the example of FIG. 7 , the userobserves the number 6 and the feedback consists in enlarging the numberidentified as being observed by applying a method for determining thefocus of visual attention according to the present description.Generally, the feedback given to the user may consist in highlightingthe identified visual stimulus, for example, by increasing itsbrightness, by making it flash, by zooming into it, by changing itsposition, by changing its size or by changing its color, etc.

In one or more embodiments, the visual stimuli form part of ahuman-machine interface of a software application or of a computationaldevice, and a command is sent to trigger the execution of one or moreoperations associated with the identified visual stimulus following theidentification of the observed visual stimulus via implementation of amethod for determining the focus of visual attention according to thepresent description.

In one or more embodiments, the various steps of the one or more methodsdescribed in this document are implemented by a software package orcomputer program.

The present description thus relates to a software package or computerprogram containing software instructions or program-code instructionsthat are readable and/or executable by a computer or by a dataprocessor, these instructions being configured to command the executionof the steps of one or more than one of the methods described in thisdocument when this computer program is executed by a computer or dataprocessor.

These instructions may use any programming language, and may take theform of source code, object code, or of code intermediate between sourcecode and object code, such as code in a partially compiled form, or inany other desirable form. These instructions are intended to be storedin a memory of a computational device or computational system, loadedthen executed by a processing unit or data processor of thiscomputational device or computational system in order to implement thesteps of one or more than one of the methods described in this document.Some or all of these instructions may be stored, temporarily orindefinitely, on a non-volatile computer-readable medium of a local orremote storing device comprising one or more storage media.

The present description also relates to a data medium readable by a dataprocessor, containing instructions of a software package or computerprogram such as mentioned above. The data medium may be any entity ordevice capable of storing such instructions. Embodiments ofcomputer-readable media comprise, without being limited thereto, bothdata-storage media and communication media comprising any medium thatfacilitates the transfer of a computer program from one location toanother. Such a storage medium may be an optical storage device such asa compact disc (CD, CD-R or CD-RW), DVD (DVD-ROM or DVD-RW) or Blu-raydisc, a magnetic medium such as a hard disk, magnetic tape or floppydisk, a removable storage medium such as a USB key, an SD or micro-SDmemory card, or even a memory, such as a random-access memory (RAM),read-only memory (ROM), cache memory, non-volatile memory, back-upmemory (for example flash or programmable memories), etc.

The present description also relates to a computational device orcomputational system comprising means for implementing the steps of oneor more than one of the methods described in this document. These meansare software and/or hardware for implementing the steps of one or morethan one of the methods described in this document.

The present description also relates to a computational device orcomputational system comprising at least one memory for storing codeinstructions of a computer program for executing all or some of thesteps of one or more than one of the methods described in this documentand at least one data processor configured to execute such a computerprogram.

What is claimed is:
 1. A computer-implemented method comprising;generating a plurality of visual stimuli using a plurality of graphicalobjects and a plurality of modulation signals; presenting the pluralityof visual stimuli in a display of a computation device to a user;recording a first plurality of electroencephalographic signals producedby the user while the user is paying attention to the displayedplurality of visual stimuli; in response to detecting ambiguity in anidentification of two or more visual stimuli of the plurality of visualstimuli using the modulation signal and the plurality ofelectroencephalographic signals, performing operations comprising:modifying an order of appearance of the plurality of visual stimuli tomore frequently display the two or more visual stimuli; recording asecond plurality of electroencephalographic signals produced by theuser; and in response to identifying one visual stimulus of the twovisual stimuli using the second plurality of electroencephalographicsignals and the plurality of modulation signals, triggering one or moreoperations of the computational device.
 2. The computer-implementedmethod of claim 1, further comprising: reconstructing a modulationsignal from the plurality of electroencephalographic signals in order toobtain a reconstructed modulation signal; computing a degree ofstatistical dependence between the reconstructed modulation signal andeach modulation signal of the plurality of modulation signals;searching, among the plurality of modulation signals, for a modulationsignal for which a degree of statistical dependence with thereconstructed modulation signal is maximal; and identifying a visualstimulus corresponding to a modulation signal for which the degree ofstatistical dependence is maximal.
 3. The computer-implemented method ofclaim 1, wherein the plurality of modulation signals are composed sothat an overall degree of statistical dependence determined in a domain,for all pairs of the plurality of modulation signals corresponding totwo separate visual stimuli, is lower than a second threshold.
 4. Thecomputer-implemented method of claim 3, wherein the domain is a timedomain.
 5. The computer-implemented method of claim 3, wherein thedomain is a frequency domain.
 6. The computer-implemented method ofclaim 2, wherein the reconstruction is carried out by applying areconstruction model to the plurality of electroencephalographicsignals.
 7. The computer-implemented method of claim 6, wherein thereconstruction model comprises a plurality of parameters of combinationsof electroencephalographic signals, and wherein the method furthercomprises determining values of parameters of the plurality ofparameters of the combinations of electroencephalographic signals in aninitial learning phase.
 8. A system comprising: at least one processor;and at least one memory storing instructions that, when executed by theat least one processor, cause the system to perform operationscomprising; generating a plurality of visual stimuli using a pluralityof graphical objects and a plurality of modulation signals; presentingthe plurality of visual stimuli in a display of a computation device toa user; recording a first plurality of electroencephalographic signalsproduced by the user while the user is paying attention to the displayedplurality of visual stimuli; in response to detecting ambiguity in anidentification of two or more visual stimuli of the plurality of visualstimuli using the modulation signal and the plurality ofelectroencephalographic signals, performing operations comprising:modifying an order of appearance of the plurality of visual stimuli tomore frequently display the two or more visual stimuli; recording asecond plurality of electroencephalographic signals produced by theuser; and in response to identifying one visual stimulus of the twovisual stimuli using the second plurality of electroencephalographicsignals and the plurality of modulation signals, triggering one or moreoperations of the computational device.
 9. The computing apparatus ofclaim 8, wherein the operations further comprise: reconstructing amodulation signal from the plurality of electroencephalographic signalsin order to obtain a reconstructed modulation signal; computing a degreeof statistical dependence between the reconstructed modulation signaland each modulation signal of the plurality of modulation signals;searching, among the plurality of modulation signals, for a modulationsignal for which a degree of statistical dependence with thereconstructed modulation signal is maximal; and identifying a visualstimulus corresponding to a modulation signal for which the degree ofstatistical dependence is maximal.
 10. The computing apparatus of claim8, wherein the plurality of modulation signals are composed so that anoverall degree of statistical dependence determined in a domain, for allpairs of the plurality of modulation signals corresponding to twoseparate visual stimuli, is lower than a second threshold.
 11. Thecomputing apparatus of claim 10, wherein the domain is a time domain.12. The computing apparatus of claim 10, wherein the domain is afrequency domain.
 13. The computing apparatus of claim 9, wherein thereconstruction is carried out by applying a reconstruction model to theplurality of electroencephalographic signals.
 14. The computingapparatus of claim 13, wherein the reconstruction model comprises aplurality of parameters of combinations of electroencephalographicsignals, and wherein the method further comprises determining values ofparameters of the plurality of parameters of the combinations ofelectroencephalographic signals in an initial learning phase.
 15. Anon-transitory computer-readable storage medium storingcomputer-executable instructions, that when executed by a computer,cause the computer to perform operations comprising; generating aplurality of visual stimuli using a plurality of graphical objects and aplurality of modulation signals; presenting the plurality of visualstimuli in a display of a computation device to a user; recording afirst plurality of electroencephalographic signals produced by the userwhile the user is paying attention to the displayed plurality of visualstimuli; in response to detecting ambiguity in an identification of twoor more visual stimuli of the plurality of visual stimuli using themodulation signal and the plurality of electroencephalographic signals,performing operations comprising: modifying an order of appearance ofthe plurality of visual stimuli to more frequently display the two ormore visual stimuli; recording a second plurality ofelectroencephalographic signals produced by the user; and in response toidentifying one visual stimulus of the two visual stimuli using thesecond plurality of electroencephalographic signals and the plurality ofmodulation signals, triggering one or more operations of thecomputational device.
 16. The non-transitory computer-readable storagemedium of claim 15, wherein the operations further comprise:reconstructing a modulation signal from the plurality ofelectroencephalographic signals in order to obtain a reconstructedmodulation signal; computing a degree of statistical dependence betweenthe reconstructed modulation signal and each modulation signal of theplurality of modulation signals; searching, among the plurality ofmodulation signals, for a modulation signal for which a degree ofstatistical dependence with the reconstructed modulation signal ismaximal; and identifying a visual stimulus corresponding to a modulationsignal for which the degree of statistical dependence is maximal. 17.The non-transitory computer-readable storage medium of claim 15, whereinthe plurality of modulation signals are composed so that an overalldegree of statistical dependence determined in a domain, for all pairsof the plurality of modulation signals corresponding to two separatevisual stimuli, is lower than a second threshold.
 18. The non-transitorycomputer-readable storage medium of claim 17, wherein the domain is atime domain.
 19. The non-transitory computer-readable storage medium ofclaim 17, wherein the domain is a frequency domain.
 20. Thenon-transitory computer-readable storage medium of claim 16, wherein thereconstruction is carried out by applying a reconstruction model to theplurality of electroencephalographic signals.